Joseph Caldwell (1773-1835), the son of Rachel Harker and physician Joseph Caldwell, was born in New Jersey and educated in New Jersey and Pennsylvania schools before entering the College of New Jersey (Princeton). In 1791 he graduated as the salutatorian at age eighteen. While studying for the ministry, Caldwell taught in a classical school in Elizabethtown, NJ, then as a tutor at the College of New Jersey. In 1795 he was licensed to preach by the Presbytery of New Brunswick, and the following year he moved to Chapel Hill to become president of the University. He married Susan Rowan of Fayetteville, NC, in 1804, but she and an infant daughter died in 1807. In 1809 Caldwell married Helen Hogg Hooper, mother of professor of ancient languages William Hooper. Caldwell received DD degrees from both the University of North Carolina and the College of New Jersey in 1816. He raised funds to complete South Building, purchased books and scientific apparatus for the University, built an observatory, and promoted public education throughout the state. He published A Compendious System of Elementary Geometry (1822); the Letters of Carlton (1828), which argue for internal improvements in North Carolina; and Letters on Popular Education Addressed to the People of North Carolina (1832).

Sources claim that in about 1829 Caldwell contracted a "chronic disease," apparently kidney stones. In 1833 he visited doctors in Philadelphia, who pronounced the disease incurable and "advised against lithotomy" (Battle 1:353), surgery to remove the stones from the bladder. Chronically ill and in considerable pain after 1833, Caldwell delegated many daily duties to other faculty members, especially Elisha Mitchell.

Professor Walker Anderson, eulogizing Caldwell during the 1835 Commencement describes the last three days of Caldwell's life: "By the exercise of prayers and other acts of the holy religion he professed, he strengthened himself for the last conflict, and spoke words of consolation and hope to his sorrowing friends. But death was yet to be indulged with a brief triumph, and for three days his sufferings were protracted with such intensity that his vigorous and well-balanced mind sank beneath the contest" (Battle 1:416). He was buried on the University campus (Dictionary of North Carolina Biography 1:303-04).